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Charter School Achievement 
on the 
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Executive Summary 



The 2003 administration of the National Assessment of Educational Progress (NAEP) in math and 
reading in grades 4 and 8 represented the first time that a nationally representative sample of charter 
schools (grade 4) was included in NAEP. This initiative was the result of a May 2002 resolution by the 
National Assessment Governing Board (NAGB) and also involved administering a special Charter School 
Survey Questionnaire to the national sample of charter schools in 2003 NAEP. As NAGB’s then 
executive director noted, “Charter schools were an important public policy issue on which NAEP could 
provide important information.” 1 Indeed, the No Child Left Behind Act (NCLB) that was signed into law 
in January 2002 included being restructured as a charter school as one of the sanctions for public schools 
that chronically fail to make “adequate yearly progress” (AYP) in achievement. 

As this year’s state lists of schools that failed to make AYP are being generated, it is already clear that 
many public schools face the possibility of being restructured as a charter school. Solid achievement data 
on charter schools are therefore more imperative than ever, and NAEP data are considered the “gold 
standard.” However, NAGB and NCES did not plan to include the 2003 NAEP charter school results in 
the 2003 NAEP report, which was released on Nov. 13, 2003. Instead, their plan called for a separate 
NAEP Charter School Report, which the National Center for Education Statistics (NCES) scheduled for 
release in January 2004 and then, according to documents from NAGB’s Nov. 14, 2003 meeting, 
postponed to June 2004. 

On March 5, 2004, NCES presented the 2003 NAEP charter school results to NAGB members at a closed 
session (permitted by law) of their meeting. The release date for the NAEP Charter School Report was 
still listed as June 2004. By NAGB’s May 2004 meeting, however, not only had the release date been 
postponed again, to December 2004, but the plan for the much-anticipated report had been fundamentally 
altered. Whereas official NAEP reports have always contained only descriptive data - which was the 
original plan for the NAEP Charter School Report, as well - NCES now proposed accompanying the 
charter school results with a special, sophisticated analysis that “would try to determine whether the 
characteristics of charter schools, such as their governance, can explain any achievement differences from 
other public schools beyond those accounted for by the characteristics of their students.” 2 

Although NAGB approved the new plan for the NAEP Charter School Report, NAGB policy (1989, 

1994) prohibits officially reporting NAEP scores with officially prepared “adjusted” or “predicted” results 
because they “would be subject to serious methodological and political challenges and would be contrary 
to the strong national commitment to encouraging high standards for all children.” 3 As then NAGB 
member Chester A. Finn said, according to NAGB minutes, “while it was proper for researchers to 
prepare adjusted scores, it would be wrong for them to [sic] part of a government report, such as NAEP. 
He said such scores would damage the credibility of program [sic].” 4 



1 NAGB, Reporting and Dissemination Committee, Report of Aug. 2, 2002. 

2 Report of May 14, 2004, NAGB Reporting and Dissemination Committee; also see April 30, 2004, NCES memo, 
“Plans for Reporting Private School and Charter School Results.” 

3 NAGB, Resolution on Reporting State -Level NAEP Results, March 5, 1994. 

4 NAGB, Reporting and Dissemination Committee, Report of March 4, 1994. 
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The issue is not the merits of a special explanatory analysis. Rather, the issue is further delaying the 
release of even the basic 2003 NAEP charter school results ostensibly for the sake of preparing such an 
analysis for inclusion in the NAEP Charter School Report (if that is even NCES’s or NAGB’s reason for 
the additional delay), especially since NAGB policy prohibits this form of official NAEP reporting. More 
important, as public schools across the nation face being restructured as a charter school because of 
NCLB’s premise that doing so would improve their performance, surely the interests of children are better 
served by a timely, “gold standard” report on charter school achievement than by waiting for an analysis 
that tries to determine whether charter school governance explains any differences in performance 
between charter and regular public schools. 

Frustrated by the repeated delays in the release of the NAEP Charter School Report and knowing that the 
data were collected in 2003, the American Federation of Teachers (AFT) decided to try to unearth the 
basic NAEP charter school results. Embedded in the questionnaire that was administered to schools along 
with the 2003 NAEP math and reading tests in grades 4 and 8 is the question: What type of school is this? 
“Charter school” was one of the possible answers. This enabled the American Federation of Teachers 
(AFT) to comb through the Web-based NAEP Data Tool to identify NAEP’s first-time, nationally 
representative sample of charter schools (grade 4) that is the subject of the inexplicably twice-delayed 
charter school report. We were similarly able to find grade 8 math and reading achievement outcomes for 
schools that identified themselves as charter schools, though this smaller sample yielded more limited 
findings than for grade 4. As a result, we were able to analyze and present our findings on grade 4 and 
grade 8 achievement in charter schools in the typical way NAEP publicly reports its results. 

We also compare charter schools to regular public schools in states where there were enough NAEP data 
to permit statistically reliable comparisons (Arizona, California, Colorado, District of Columbia, 

Michigan and Texas). Because charter schools in Arizona, the District of Columbia, Michigan and Texas 
have more freedom from the rules that ordinarily govern public schools than do charters in California and 
Colorado, it was possible to make an initial assessment of how governance is related to achievement. That 
question is the focus of the explanatory analysis that will be part of the official report of the NAEP charter 
school results, now scheduled for December 2004. 

Unfortunately, the NAEP Data Tool only permits a comparison between charter schools and other public 
schools by the specific factors presented in our study and not by the dozens of other student, school and 
community characteristics that NAEP gleans. Even more significantly, the NAEP Data Tool does not 
contain the detailed Charter School Survey Questionnaire that was administered for the 2003 NAEP 
charter school report, so its results could not be analyzed at this time. 

How Did Charter School Students Perform on 2003 NAEP? 

• Average scores. Compared to students in regular public schools, charter school students had lower 
achievement both in grade 4 (six scale points lower in math, seven scale points lower in reading) and 
grade 8 (five points lower in math, two points lower in reading). These differences were all 
statistically significant, except for grade 8 reading, and translate into about a half year of schooling. 

• Achievement levels. In grades 4 and 8 and both in math and reading, the percentages of charter 
school students performing at or above Basic and at or above Proficient were lower than the 
corresponding percentages for regular public school students. 

Free or Reduced-Price Lunch Eligibility. Because the evidence suggests that charter schools are a little 
more likely than other public schools to enroll poor children, a fair comparison between the two kinds of 
public schools must include comparing students with similar economic backgrounds. The NAEP Data 
Tool only permits doing so by eligibility for free or reduced-price lunch, which is a frequently used proxy 
for economic background. 



This document is available at the EPSL website at http://www.asu.edu/educ/epsl/EPRU/articles/EPRU-0408-63-OWI.pdf 



11 




• The scores of students who were eligible for free or reduced-price lunch, as well as the scores of 
students who were not, were lower in charter schools than in regular public schools both in grades 4 
and 8 and both for math and reading. Among lunch-eligible students, the statistically significant 
difference was nearly six scale points in grade 4 math and seven points in reading. These differences 
translate into a little more than a half year of schooling. In grade 8, the difference between charter and 
regular public schools was nearly seven scale points in math and four points in reading, but only the 
math result was statistically significant. 

• Achievement gap. In both kinds of public schools, the achievement gap between students who were 
and were not eligible for free or reduced-price lunch was similarly substantial in both subjects and 
both grades, but the gap was slightly larger in charter schools than in regular public schools in grade 4 
reading and grade 8 math. 

Central-City Location. Charter school operators often locate in or near central cities, where regular 
public schools are under fire and parents more likely to seek education alternatives for their youngsters. 
Because student achievement is generally lower in central cities, it is therefore important to ask whether 
or not the lower student achievement in charter schools reflects the greater likelihood of charter school 
location in central cities. The comparison of central-city charter schools with central-city regular public 
schools suggests it does not. (Note: Only grade 4 results are presented because the grade 8, central-city 
charter school sample was too small to compare reliably to the regular public school sample.) 

• Both in terms of average scores and achievement levels for math and reading alike, regular public 
schools in central cities outperformed charter schools in those locations. The statistically significant 
math difference was a little more than a half year of schooling, while in reading it was a little less 
than a half year. 

Minority Students. Given the predominantly central-city location of charter schools, their higher 
percentage of black students (33 percent, grade 4) compared to the statewide percentage for other public 
schools (18 percent, grade 4) is not surprising. Charters, however, are no more likely than regular public 
schools to enroll Hispanic students. Because minority student achievement is generally low, it is therefore 
important to ask whether or not charters’ disproportionate enrollment of black (but not Hispanic) students 
explains the lower achievement of charter schools relative to regular public schools. The NAEP results 
suggest it does not. (Note: Results are presented only for grade 4 because the grade 8 NAEP charter 
school sample does not allow for statistically reliable comparisons by race and ethnicity.) 

• Compared to their peers in regular public schools, black and Hispanic charter school students scored 
lower both in math and reading in grade 4, but the differences were not statistically significant. The 
achievement gaps between white and black students and between white and Hispanic students were 
about the same in charter schools as in regular public schools. 

Achievement of More versus Less State-Regulated Charter Schools. Because the results of the 
special NAEP Charter School Survey Questionnaire are not available on the NAEP Data Tool, we could 
not conduct a detailed analysis of the governance question that is the focus of the unprecedented 
explanatory analysis that will be part of the official report of the NAEP charter school results currently 
scheduled for December 2004. However, it is possible to offer preliminary evidence about whether 
differences in the degree of state regulation of charter schools are associated with differences in charter 
school performance by examining Arizona, California, Colorado, the District of Columbia, Michigan and 
Texas. (Note: The NAEP Data Tool did not provide charter school data for other states either because the 
sample was too small or the state had no charter schools. Moreover, only results from the nationally 
representative grade 4 charter school sample are reported because the size of the grade 8 charter school 
sample is insufficient to permit reliable comparisons among the states.) 

• The interstate analysis of charter school governance and student achievement undercuts the idea that 
more charter school autonomy produces higher student achievement among charter schools and 
between charters and regular public schools. 



iii 
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• In Arizona and the District of Columbia, both of which have very autonomous charter schools, charter 
schools underperformed regular public schools, but not at statistically significant levels. In the two 
other states with very autonomous charter schools, Michigan and Texas, charter schools performed 
significantly lower than regular public schools. In math, the difference in both states was equivalent 
to about a year of schooling. In reading, Michigan’s charter schools underperformed regular public 
schools by the equivalent of about two years of schooling, and in Texas, it was about one and a half 
years. 

• In the two states with the least autonomous charter schools, California and Colorado, charter schools 
and regular public schools scored about the same. 
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Background (see Appendix A for a chronology of events) 

The National Assessment of Educational Progress (NAEP), often termed “the nation’s report card,” has 
been testing the academic achievement of a nationally representative sample of students and publicly 
reporting the results since 1969. NAEP is a project of the National Center for Education Statistics 
(NCES), which is within the Institute of Education Sciences (IES) of the U.S. Department of Education. 
Overall policy direction for NAEP is the responsibility of the National Assessment Governing Board 
(NAGB), an independent entity whose members are appointed by the U.S. secretary of education 
according to categories set by Congress. 1 

In 2003, NAEP conducted national and state assessments in reading and mathematics in grades 4 and 8. 
This marked the first time that a nationally representative sample of charter schools (grade 4) was part of 
NAEP. Those schools also received a special NAEP Charter School Survey Questionnaire, which NAGB 
had approved on Nov. 16, 2002. Grade 8 charter schools were also included in 2003 NAEP, but these 
schools, unlike the grade 4 sample, were not specially drawn to be nationally representative. 

Including a nationally representative sample of charter schools in 2003 NAEP was the result of a May 
2002 NAGB resolution that had been initiated “by a group of organizations that are interested in the 
developing (charter school) movement.” 2 As then NAGB executive director Roy Truby noted, “Charter 
schools were an important public policy issue on which NAEP could provide important information.” 3 
Indeed, by that time there were already about 2,000 charter schools nationwide. Moreover, policymakers 
often found themselves caught between researchers’ findings that there was little or nothing to distinguish 
charter school performance from that of comparable, regular public schools and advocacy groups’ 
opposing claims. Further underscoring the public policy significance of the charter school movement was 
the fact that the No Child Left Behind Act (NCLB), which was signed into law on Jan. 8, 2002, included 
being restructured as a charter school as one of the sanctions for public schools that chronically fail to 
make “adequate yearly progress” in achievement. With charter schools now a key part of high-stakes 
accountability, NAGB’s 2002 decision to examine and report their performance on NAEP - often dubbed 
the “gold standard” of assessment - could not have been timelier. 

The 2003 NAEP results were released on Nov. 13, 2003, but the charter school results were not among 
them. NAGB had instead planned to produce a separate NAEP Charter School Report, which NCES 
originally scheduled for release in January 2004. However, one day after the Nov. 13 release of 2003 
NAEP, NCES informed NAGB members during their board meeting that the Charter School Report 
would be delayed to June 2004. 

On March 5, 2004, NCES presented the 2003 NAEP charter school results to NAGB members at a closed 
session (permitted by law) of their meeting. The release date for the NAEP Charter School Report was 
still listed as June 2004. By NAGB’s May 2004 meeting, however, not only had the release date been 
postponed again, to December 2004 - more than a year after the other 2003 NAEP results had been 
released — but the plan for the report had been fundamentally altered. Whereas official NAEP reports 
have always contained only descriptive data - which was the original plan for the NAEP Charter School 
Report, as well - NCES now proposed accompanying the charter school results with a special analysis, 
using hierarchical linear modeling (HLM), 4 that “would try to determine whether the characteristics of 



1 The NCES Commissioner is Robert Lerner, whose nomination was sent to the U.S. Senate by President George 
Bush on June 3, 2003, and whose recess appointment was announced on Dec. 26, 2003. NAGB’s chair is Darvin 
Winick, who was selected for the position by U.S. Secretary of Education Rod Paige in November 2002. 

2 NAGB, Reporting and Dissemination Committee, Report of Aug. 2, 2002. 

3 Ibid. 

4 The following description of HLM is abstracted from Anthony S. Bryk and Stephen W. Raudenbush, Hierarchical 
Linear Models for Social and Behavioural Research: Applications and Data Analysis Methods (Newbury Park, 
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